6 Competitive Networks and Competitive Learning

Competitive neural networks belong to a class of recurrent networks, and-they are based on algorithms of unsupervised learning, such as the competitive algorithm explained in this section. In competitive learning, the output neurons of a neural network compete among themselves to become active (to be "fired"). Whereas in multiplayer perceptrons several output neurons may be active simultaneously, in competitive learning only a single output neuron is active at anyone time. There are three basic elements necessary to build a network with a competitive learning rule, a standard technique for this type of artificial neural networks: 1. A set of neurons that have the same structure and that are connected with initially randomly selected weights. Therefore, the neurons respond differently to a given set of input samples. 2. A limit value that is determined on the strength of each neuron. 3. A mechanism that permits the neurons to complete for the right to respond to a given subset of inputs, such that only one output neuron is active at a time. The neuron that wins the competition is called winner-takes-all neuron. In the simplest form of competitive learning, an ANN has a single layer of output neurons, each of which is fully connected to the input nodes. The network may include feedback connections among the neurons, as indicated in Figure 1. In the network architecture described herein, the feedback connections perform lateral inhibition, with each neuron tending to inhibit the neuron to which it is laterally connected. In contrast, the feedforward synaptic connections in the net-work of Figure 9.12 are all excitatory.